1. Field of the Disclosure
The present disclosure relates to implementing learning in spiking neuron networks.
2. Description of Related Art
Spiking Neural Networks
Artificial spiking neural networks may be frequently used to gain an understanding of biological neural networks, and/or for solving artificial intelligence problems. These networks may typically employ a pulse-coded mechanism, which may encode information using timing of the pulses. Such pulses (also referred to as “spikes” or ‘impulses’) may be short-lasting (typically on the order of 1-2 ms) discrete temporal events. Several exemplary embodiments of such encoding are described in a commonly owned and co-pending U.S. patent application Ser. No. 13/152,084 filed Jun. 2, 2011, and entitled APPARATUS AND METHODS FOR PULSE-CODE INVARIANT OBJECT RECOGNITION”, and U.S. patent application Ser. No. 13/152,119 filed Jun. 2, 2011, and entitled “SENSORY INPUT PROCESSING APPARATUS AND METHODS”, each being incorporated herein by reference in its entirety.
A typical artificial spiking neural network, such as the network 100 shown for example in FIG. 1, may comprise a plurality of units (or nodes) 102, which may correspond to neurons in a biological neural network. Any given unit 102 may receive input via connections 104, also referred to as communications channels, or synaptic connections. Any given unit 102 may further be connected to other units via connections 112, also referred to as communications channels, or synaptic connections. The units (e.g., the units 106 in FIG. 1) providing inputs to any given unit via for example connections 104, are commonly referred to as the pre-synaptic units, while the unit receiving the inputs (e.g., the units 102 in FIG. 1) is referred to as the post-synaptic unit. Furthermore, the post-synaptic unit of one unit layer (e.g. the units 102 in FIG. 1) may act as the pre-synaptic unit for the subsequent upper layer of units (not shown).
Individual ones of the connections (104, 112 in FIG. 1) may be assigned, inter alia, a connection efficacy (which in general may refer to a magnitude and/or probability of influence of pre-synaptic spike to firing of post-synaptic neuron, and may comprise, for example a parameter: synaptic weight, by which one or more state variables of post synaptic unit are changed). During operation of the pulse-code network (e.g., the network 100), synaptic weights may be typically adjusted using what is referred to as the spike-timing dependent plasticity (STDP) in order to implement, among other things, network learning.
One such adaptation mechanism is illustrated with respect to FIGS. 2-3. Traces 200, 210 in FIG. 2 depict pre-synaptic input spike train (delivered for example via connection 104_1 in FIG. 1) and post synaptic output spike train (generated, for example, by the neuron 102_1 in FIG. 1), respectively.
Properties of the connections 104 (such as weights w) may be typically adjusted based on relative timing between the pre-synaptic input (e.g., the pulses 202, 204, 206, 208 in FIG. 2) and post-synaptic output pulses (e.g., the pulses 214, 216, 218 in FIG. 2). One typical STDP weight adaptation rule is illustrated in FIG. 3, where rule 300 depicts synaptic weight change Δw as a function of time difference between the time of post-synaptic output generation and arrival of pre-synaptic input Δt=tpost−tpre. In some implementations, synaptic connections (e.g., the connections 104 in FIG. 1) delivering pre-synaptic input prior to the generation of post-synaptic response may be potentiated (as indicated by Δw>0 associated with the curve 304 in FIG. 3), while synaptic connections (e.g., the connections 104 in FIG. 1) delivering pre-synaptic input subsequent to the generation of post-synaptic response may be depressed). By way of illustration, when the post-synaptic pulse 208 in FIG. 2 is generated: (i) connection associated with the pre-synaptic input 214 precedes the output pulse (indicated by the line denoted 224) and it may be potentiated (Δw>0 in FIG. 3 and the weight is increased); and (ii) connections associated with the pre-synaptic input 216, 218 that follow may be depressed.
Spiking Neuron Models
Generalized dynamics equations for spiking neurons models may be often expressed as a superposition of input, interaction between the input current and the neuronal state variables, and neuron reset after the spike as follows:
                    ⅆ                  q          →                            ⅆ        t              =                  V        ⁡                  (                      q            →                    )                    +                        ∑                      t            out                          ⁢                              R            ⁡                          (                              q                →                            )                                ⁢                      δ            ⁡                          (                              t                -                                  t                  out                                            )                                          +                        G          ⁡                      (                          q              →                        )                          ⁢                  I          ext                      ,where:                {right arrow over (q)} is a vector of internal state variables (e.g., comprising membrane voltage);        Iext is external input into neuron;        V is the function that defines evolution of the state variables;        G describes the interaction between the input current and the state variables (for example, to model postsynaptic potentials); and        R describes resetting the state variables after the output spikes at tout.        
For example, for IF model the state vector and the state model may be expressed as:{right arrow over (q)}(t)≡u(t); V({right arrow over (q)})=−Cu; R({right arrow over (q)})=ures; G({right arrow over (q)})=1,  (Eqn. 2)where C—membrane constant, ures—value to which voltage is set after output spike (reset value). Accordingly, Eqn. 1 becomes:
                                          ⅆ            u                                ⅆ            t                          =                              -            Cu                    +                                    ∑                              t                out                                      ⁢                                          (                                                      u                    refr                                    -                  u                                )                            ⁢                              δ                ⁡                                  (                                      t                    -                                          t                      out                                                        )                                                              +                                    I              ext                        .                                              (                  Eqn          .                                          ⁢          3                )            
For the simple neuron model described by Izhikevich E. M., 2003, Eqn. 1 may be expressed as:
                                                                        ⅆ                v                                            ⅆ                t                                      =                                          0.04                ⁢                                  v                  2                                            +                              5                ⁢                v                            +              140              -              u              +                                                ∑                                      t                    out                                                  ⁢                                                      (                                          c                      -                      v                                        )                                    ⁢                                      δ                    ⁡                                          (                                              t                        -                                                  t                                                      out                            ⁢                                                                                                                                                                                    )                                                                                  +                              I                ext                                              ,                                          ⁢                                          ⁢          and                ⁢                                  ⁢                                  ⁢                                                            ⅆ                u                                            ⅆ                t                                      =                                          -                                  a                  ⁡                                      (                                          bv                      -                      u                                        )                                                              +                              d                ⁢                                                      ∑                                          t                      out                                                        ⁢                                      δ                    ⁡                                          (                                              t                        -                                                  t                          out                                                                    )                                                                                                    ,                                          ⁢                                          ⁢                      where            ⁢                          :                                                          (                  Eqn          .                                          ⁢          4                )                                                          ⁢                                                                              q                  →                                ⁡                                  (                  t                  )                                            ≡                              (                                                                                                    v                        ⁡                                                  (                          t                          )                                                                                                                                                                        u                        ⁡                                                  (                          t                          )                                                                                                                    )                                      ;                                          V                ⁡                                  (                                      q                    →                                    )                                            =                              (                                                                                                                              0.04                          ⁢                                                                                    v                              2                                                        ⁡                                                          (                              t                              )                                                                                                      +                                                  5                          ⁢                                                      v                            ⁡                                                          (                              t                              )                                                                                                      +                        140                        -                                                  u                          ⁡                                                      (                            t                            )                                                                                                                                                                                                  a                        ⁡                                                  (                                                                                    bv                              ⁡                                                              (                                t                                )                                                                                      -                                                          u                              ⁡                                                              (                                t                                )                                                                                                              )                                                                                                                    )                                      ;                    ⁢                                          ⁢                                          ⁢                                                    R                ⁡                                  (                                      q                    →                                    )                                            =                              (                                                                                                    c                        -                                                  v                          ⁡                                                      (                            t                            )                                                                                                                                                                          d                                                                      )                                      ;                                          G                ⁡                                  (                                      q                    →                                    )                                            =                              (                                                                            1                                                                                                  0                                                                      )                                                                        (                  Eqn          .                                          ⁢          5                )            and a,b,c,d are parameters of the model.
Some algorithms for spike-time learning (especially, reinforcement learning) in spiking neural networks are typically represented using the following general equation described, for example, in co-pending and co-owned U.S. patent application Ser. No. 13/489,280, filed Jun. 5, 2012, 2012, entitled “APPARATUS AND METHODS FOR REINFORCEMENT LEARNING IN ARTIFICIAL SPIKING NEURAL NETWORKS”, incorporated supra:
                                                        ⅆ                                                θ                  i                                ⁡                                  (                  t                  )                                                                    ⅆ              t                                =                      η            ⁢                                                  ⁢                          F              ⁡                              (                t                )                                      ⁢                                          e                i                            ⁡                              (                t                )                                                    ,                            (        6        )            where:                θi(t) is an adaptation (learning) parameter of a synaptic connection between the pre-synaptic neuron i and the post-synaptic neuron j;        η is a parameter referred to as the learning rate, η can be a constant parameter or it can be a function of some other system parameters;        F(t) is a performance function; and        ei(t) is eligibility trace, configured to characterize relations between pre-synaptic and post-synaptic activity.        
By way of background, an exemplary eligibility trace may comprise a temporary record of the occurrence of an event, such as visiting of a state or the taking of an action, or a receipt of pre-synaptic input. The trace marks the parameters associated with the event (e.g., the synaptic connection, pre- and post-synaptic neuron IDs) as eligible for undergoing learning changes. In one approach, when a reward signal occurs, only eligible states or actions may be ‘assigned credit’ or ‘blamed’ for the error. Thus, the eligibility traces aid in bridging the gap between the events and the training information.
In some approaches, such as described for example in a co-owned and co-pending U.S. patent application Ser. No. 13/489,280 entitled “APPARATUS AND METHODS FOR REINFORCEMENT LEARNING IN ARTIFICIAL NEURAL NETWORKS”, filed Jun. 5, 2012, incorporated supra. The above reference application may describe synapses of the neuron by a linear dynamic process, characterized by a state vector {right arrow over (S)}i:
                                                        ⅆ                                                                    S                    →                                    i                                ⁡                                  (                  t                  )                                                                    ⅆ              t                                =                                                    -                                  A                  ⁡                                      (                                          q                      ,                      t                      ,                                              t                        out                                                              )                                                              ⁢                                                                    S                    →                                    i                                ⁡                                  (                  t                  )                                                      +                                                            k                  →                                i                            ⁢                                                ∑                  j                                ⁢                                  δ                  ⁡                                      (                                          t                      -                                              t                        j                                                  i                          ⁢                                                                                                          ⁢                          n                                                                                      )                                                                        +                                          ∑                                  t                  out                                            ⁢                                                Rs                  ⁡                                      (                    t                    )                                                  ⁢                                  δ                  ⁡                                      (                                          t                      -                                              t                        out                                                              )                                                                                      ,                            (        7        )            where:                A(t) is a stable matrix, dependent on the state q of the neuron, output spike time tout, and/or time;        Σj δ(t−tjin) is the term describing the input spike train;        {right arrow over (k)}i is vector coefficient characterizing individual synapses and effect of the presynaptic spikes on it; and        Rs(t) describes resetting the synapse state vector {right arrow over (S)}i after the output spikes at tout.Synaptic state variable can be subject to reset depending on neuron firing times. Synaptic state can be used as a component in different neuronal mechanisms, such as short-term and long-term plasticity.        
For example, as described in detail in U.S. patent application Ser. No. 13/560,902 entitled “APPARATUS AND METHODS FOR GENERALIZED STATE-DEPENDENT LEARNING IN SPIKING NEURON NETWORKS”, filed Jul. 27, 2012 incorporated supra, eligibility traces ei(t) of Eqn. 6 for a broad spectrum of long-term plasticity learning rules may be expressed as:
                                                        ⅆ                                                e                  i                                ⁡                                  (                  t                  )                                                                    ⅆ              t                                =                                    -                                                                    e                    i                                    ⁡                                      (                    t                    )                                                  τ                                      +                                          Q                ⁡                                  (                  t                  )                                            ⁢                                                S                  ik                                ⁡                                  (                  t                  )                                                                    ,                            (                  Eqn          .                                          ⁢          8                )            where:                τ is a time constant,        Q(t) is a function of a state of the neuron q that is the same for individual synapses, and        Sik(t) is the k-th element in the synaptic state vector {right arrow over (S)}i.        
In spiking neurons simulations Eqn. 7-Eqn. 8 may be typically solved using numerical integration techniques such as Euler integration rule:Si(t+1)=Si(t)−Δt[ASi(t)−kiδ(t−tiin)], and  (Eqn. 9)ei(t+1)=ei(t)−Δt[ei(t)/τ−Q(t)Sik(t)],  (Eqn. 10)where:                δ(t−tiin)=1 for t=tiin; and 0 otherwise,        δt is the update time step.In some implementations, the post-synaptic spike synapse state traces may be subject to a reset condition upon generation of the post-synaptic spike. In one or more implementations, the reset may be implemented by determining the state trace value ei in accordance, for example, with Eqn. 10. In some implementations, the value of the state parameter Si (taken prior to the reset) may be used be used in Eqn. 10 in order to determine ei as follows:Si(t+1)=Rs(Si(t))δ(t−tout,  (Eqn. 11)where Rs is the reset function.        
By way of illustration, when a number of computations to be performed for individual synapses at individual update time steps is C, then for a neuron interfaced to NS synapses, the total number of computations (per time step) becomes NC=C×NS. In order to perform adequately complex neural network simulations, the number of synapses per neuron NS may be typically configured between 100 and 100,000. Hence, in order to evaluate neuronal state at 1 ms intervals (the update rate frequently used in the arts), the required number of computations per neuron per time step may be NC=C×[100−100,000] per neuron per time step. Such high computational load may place demanding requirements on computational power of the neuromorphic hardware. Furthermore, the computational load scales with the number of synapses, thus severally limiting number of neurons that may be operated by the neuromorphic processing and/or frequency of network updates which may negatively affect network precision.
Accordingly, there may be a salient need for a more efficient method and apparatus for implementing synaptic updates in spiking neural network comprising medium to high density of synapses.